Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add exception handling to task running #34

Open
wants to merge 2 commits into
base: develop
Choose a base branch
from
Open

Add exception handling to task running #34

wants to merge 2 commits into from

Conversation

alexgleith
Copy link
Contributor

Not sure if this is naive, but I have some tasks failing, and Statistician currently crashes, rather than marking the task as an error and continuing.

I think this is the right place to catch exceptions and gracefully return an error notification.

@codecov
Copy link

codecov bot commented Mar 30, 2022 •
edited
Loading

Codecov Report

Merging #34 (7643227) into develop (9ee6375) will decrease coverage by 0.12%.
The diff coverage is 0.00%.

@@             Coverage Diff             @@
##           develop      #34      +/-   ##
===========================================
- Coverage    62.01%   61.88%   -0.13%     
===========================================
  Files           20       20              
  Lines         1935     1939       +4     
===========================================
  Hits          1200     1200              
- Misses         735      739       +4
Impacted Files Coverage Δ
odc/stats/proc.py 26.00% <0.00%> (-0.54%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9ee6375...7643227. Read the comment docs.

supermarkion
supermarkion approved these changes Mar 31, 2022
View reviewed changes
Copy link
Contributor

@supermarkion supermarkion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be worth to have a try. Better than no try-catch.

emmaai
emmaai reviewed Mar 31, 2022
View reviewed changes
Copy link
Contributor

@emmaai emmaai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have my doubt on this "catching everything then move on" approach. And I'm not sure about the context, if it's the only resort to fix the problem, then yes, if we could do more work to avoid crash, then we should.

@alexgleith
Copy link
Contributor Author

I have my doubt on this "catching everything then move on" approach. And I'm not sure about the context, if it's the only resort to fix the problem, then yes, if we could do more work to avoid crash, then we should.

I agree that crashes shouldn't happen, but if they do, we have a choice of how we handle them. Currently, it's basically by aborting the whole job. What should happen is that Task should fail, be logged and the next Task can be tried.

This is especially important for streaming from SQS, which has built-in retries and failed task logging using the deadletter queue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@emmaai emmaai emmaai left review comments

@supermarkion supermarkion supermarkion approved these changes

@Kirill888 Kirill888 Awaiting requested review from Kirill888

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@alexgleith @emmaai @supermarkion